|
DRBD - Distributed Storage
2011/06/02 |
|
Install DRBD(Distributed Replicated Block Device) and configure Distributed Storage System.
This example shows to configure on the environment like follows.
(1) www01.srv.world [10.0.0.50]
(2) www02.srv.world [10.0.0.51]
It needs your machine has enough area to use for free, if your computer does not have such drive, add drive in your computer.
|
|
| [1] | Install DRBD on both hosts. When installing, a kernel is also installed for dependency, but it is not a latest version, the DRBD from ATrpms depends on "kernel-2.6.32-71.7.1.el6.x86_64" (June 2011 now), so if your machine has laso latest kernel, please downgrade to "kernel-2.6.32-71.7.1.el6.x86_64" to use DRBD from ATrpms. |
|
# reboot to run with a kernel for DRBD [root@www01 ~]# |
| [2] | Create a LVM partition for DRBD device on both hosts. After creating it, create a logical volume like follows. |
|
[root@www01 ~]# pvcreate /dev/sdb1 Physical volume "/dev/sdb1" successfully created [root@www01 ~]# vgcreate vg_r0 /dev/sdb1 Volume group "vg_r0" successfully created [root@www01 ~]# lvcreate -L 80G -n lv_r0 vg_r0 Logical volume "lv_r0" created |
| [3] | Configure DRBD. Set them on both hosts. |
|
[root@www01 ~]#
vi /etc/drbd.d/global_common.conf
disk {
# line 27: add ( detach disks when IO-error )
on-io-error detach;
syncer {
# line 38: add ( band width for synchronization )
rate 300M;
[root@www01 ~]#
vi /etc/drbd.d/r0.res # create new
resource r0 { # DRBD device device /dev/drbd0; # phisical device disk /dev/vg_r0/lv_r0; meta-disk internal; on www01.srv.world { # IP address:port address 10.0.0.50:7788; } on www02.srv.world { address 10.0.0.51:7788; } } modprobe drbd # load DRBD module [root@www01 ~]# drbd 286064 0
[root@www01 ~]#
drbdadm create-md r0 # create DRBD resource --== Thank you for participating in the global usage survey ==-- The server's response is: you are the 7635th user to install this version Writing meta data... initializing activity log NOT initialized bitmap New drbd meta data block successfully created. success[root@www01 ~]# /etc/rc.d/init.d/drbd start
Starting DRBD resources: [
[root@www01 ~]# r0 Found valid meta data in the expected location, 85899341824 bytes into /dev/vg_r0/lv_r0. d(r0) s(r0) n(r0) ].......... *************************************************************** DRBD's startup script waits for the peer node(s) to appear. - In case this node was already a degraded cluster before the reboot the timeout is 0 seconds. [degr-wfc-timeout] - If the peer was available before the reboot the timeout will expire after 0 seconds. [wfc-timeout] (These values are for resource 'r0'; 0 sec -> wait forever) To abort waiting enter 'yes' [ 18]: yes
chkconfig drbd on [root@www01 ~]# echo "/sbin/modprobe drbd" >> /etc/rc.local |
| [4] | Make a Host be primary, then synchronization starts. |
|
[root@www01 ~]# cat /proc/drbd
version: 8.3.8.1 (api:88/proto:86-94)
GIT-hash: 0d8589fcc32c874df57c930ca1691399b55ec893 build by gardner@, 2011-01-21 17:26:47
0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r----
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:83883484
[root@www01 ~]# drbdsetup /dev/drbd0 primary -o # set primary [root@www01 ~]# cat /proc/drbd
version: 8.3.8.1 (api:88/proto:86-94)
GIT-hash: 0d8589fcc32c874df57c930ca1691399b55ec893 build by gardner@, 2011-01-21 17:26:47
0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r----
ns:1985536 nr:0 dw:0 dr:1993240 al:0 bm:121 lo:1 pe:29 ua:221 ap:0 ep:1 wo:b oos:81898844
[>....................] sync'ed: 2.4% (79976/81916)M
finish: 0:06:11 speed: 220,512 (220,512) K/sec
# after few minutes later, synchronization completes and the status truns like follows. [root@www01 ~]# cat /proc/drbd
version: 8.3.8.1 (api:88/proto:86-94)
GIT-hash: 0d8589fcc32c874df57c930ca1691399b55ec893 build by gardner@, 2011-01-21 17:26:47
0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----
ns:83883484 nr:0 dw:0 dr:83884148 al:0 bm:5120 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
|
| [5] | Configuration completes. It's OK to create a filesystem on DRBD device and mount it on a primary host to use it. |
|
[root@www01 ~]# mkfs -t ext4 /dev/drbd0 [root@www01 ~]# mount /dev/drbd0 /mnt [root@www01 ~]# touch /mnt/test.txt # create a test file [root@www01 ~]# ll /mnt total 16 drwx------. 2 root root 16384 Jun 4 04:10 lost+found -rw-r--r--. 1 root root 0 Jun 4 04:12 test.txt |
| [6] | If you'd like to mount DRBD device on secondary Host, unmount DRBD device on primary Host first and make primary Host be secondary. Next make secondary Host be primary and mount DRBD device. |
|
########### on Primary Host ########### [root@www01 ~]# umount /mnt [root@www01 ~]# drbdadm secondary r0 # set secondary ########### on Secondary Host ########### [root@www02 ~]# drbdadm primary r0 # set primary [root@www02 ~]# mount /dev/drbd0 /mnt [root@www02 ~]# ll /mnt total 16 drwx------. 2 root root 16384 Jun 4 04:10 lost+found -rw-r--r--. 1 root root 0 Jun 4 04:12 test.txt |